docs: extend IR to the full cross-format capability surface - #3
Merged
Conversation
The IR draft could not losslessly host several constructs that the target source formats declare, which would have forced schema changes the moment those frontends landed. Close the gaps found by auditing the model against Smithy 2.0, TypeSpec (incl. http/streams/sse/events/versioning/rest/xml and the new visibility system), Protobuf (proto2/proto3/editions + gRPC transcoding), GraphQL (current spec + draft), AsyncAPI 3.0, OpenAPI 3.1/3.2, and a sweep of Thrift/Avro/JSON-RPC/WSDL/Cap'n Proto — and add Erlang/OTP (gen_server/gen_statem/gen_event message-protocol surface) as a supported frontend target. Headline model changes: - Operation.OneWay: fire-and-forget ops (OTP cast, AsyncAPI send-without- reply, Thrift oneway, JSON-RPC notifications) are distinct from "response with no body" - messages become registry citizens (MessageID, Document.Messages); AsyncAPI request-reply gains Reply with dynamic reply addresses - new OTPBinding (call/cast/info against a process channel); positional discrimination (Discriminator.Index) and Model.Positional for tagged tuples and records; symbol values for atoms - services get IDs, inheritance, versions, per-service renames, and protocol declarations; servers get names, auth, and protocol versions - OpBindings.HTTP becomes a slice (gRPC transcoding additional_bindings) - Property gains Presence (protobuf tri-state), ClientOptional/DefaultAdded (Smithy evolution model), EventPayload; WireID becomes *int (0 is a legal ordinal in Cap'n Proto/FlatBuffers/Avro) - Visibility.None (@invisible), type-level XML/wire-name/media-type hints, richer Example (input/output/error scenarios, header+payload pairs), Availability on enum members/variants/params with add/remove cycles and optionality flips - AuthKind covers the SASL/X509/userPassword families; Extensions and Provenance land on every node that can carry source metadata The capability matrix gains an Erlang/OTP column, six new capability rows, and ~25 cell corrections; lowering summaries are rewritten per format and documented conventions added for degraded constructs (never-typed members, bit-sized binaries, waiters, endpoint rules, GraphQL directive ordering).
The original prior-art survey of oagen was written from a high-level study; auditing the actual source (parser, engine, and the differ/compat/verify subsystems the survey never covered) verified every §1 claim, sharpened four that were imprecise, and surfaced new lessons. None require IR schema changes — the strongest result is negative: nothing in oagen's ~25k-line codebase needed information the IR lacks, while its compat stack repeatedly re-derives by heuristic (field-set matching, Jaccard overlay correlation) the stable spec-level identity the IR carries by design. - prior-art.md §1: verified/sharpened mistake rows (discriminated request bodies are also flattened; discriminator inference is structural but welded to derived names; unwrapping applied destructively), new rows (ref-kind-on-reference forcing global state, collision-cascade renames, name-list accretion), new worth-adopting entries (wire-identity-keyed hints, group-metadata-over-wire-list, generation manifests, spec-only smoke baselines, behavioral-change channel, narrow expiring approvals), and a new subsection on the diff/compat/verify subsystems - architecture.md: real-world plan-layer decision list; canonical policy- input vocabulary with a declared-facts-win precedence rule; named write/integrate and surface-verification stages; filter-vs-scoped- regeneration distinction; pre-IR document patching vs IR overlay vs backend overlays; allowlist shape; wire-conformance test bullet - ir-design.md: $ref-sibling merge precedence and union-of-consts normalization placement in the OpenAPI lowering row; documented promotion path for mutually-exclusive parameter groups; open question 5 gains the structural third correlation tier
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The IR draft had gaps that would have forced schema changes the moment later frontends landed — violating the core rule that only frontends are staged over time while the IR capability surface is complete from day one. This revision closes them, based on an audit of the model against the official specs for Smithy 2.0, TypeSpec (language + http/streams/SSE/events/versioning/rest/xml libraries and the current visibility system), Protobuf (proto2/proto3/editions, gRPC + transcoding), GraphQL (October 2021 + working draft), AsyncAPI 3.0, and OpenAPI 3.1/3.2, plus a representability sweep over Thrift, Avro, JSON-RPC/OpenRPC, WSDL, and Cap'n Proto. It also adds Erlang/OTP as a supported frontend target, scoped to the message-protocol surface (gen_server/gen_statem/gen_event call/cast/info, Dialyzer type language).
Headline changes to
docs/ir-design.md(normative):Operation.OneWaydistinguishes fire-and-forget (OTP cast, AsyncAPI send-without-reply, Thriftoneway, JSON-RPC notifications) from "response with no body".MessageID+Document.Messagesregistry (they were embedded by value, contradicting the no-embedded-named-nodes rule, andMessageRefwas referenced but never defined); AsyncAPI request-reply gainsReplywith static channel + dynamic reply address + reply message set; message headers become a typed schema.OTPBinding(behaviour, call/cast/info, target process as a channel, request tag); positional discrimination (Discriminator.Index) for tagged-tuple unions;Model.Positionalfor records;symbolvalues for atoms.Extends), per-service versions/renames, and protocol declarations; servers gain names, server-scoped auth (AsyncAPI's primary placement), and protocol versions.OpBindings.HTTPbecomes a slice — gRPC transcodingadditional_bindingsneeds N HTTP mappings per operation; bindings also gain transcoding path patterns,response_bodyprojection, compression/checksum obligations, and shared routes.Presence(protobuf implicit/explicit/required — not nullability),ClientOptional/DefaultAdded(Smithy evolution model),EventPayload;WireIDbecomes*int(0 is a legal ordinal in Cap'n Proto/FlatBuffers/Avro).@invisible), operation-level visibility overrides, type-level XML/wire-name/media-type hints, richerExample(input↔output/error scenarios, header+payload pairs),Availabilityon enum members/variants/parameters with add/remove cycles and optionality flips.Extensions/Provenancenow exist on every node that can carry source metadata (responses, contents, examples, flows, bindings, auth schemes).never/string templates, Erlang bit-sized binaries and funs, Smithy waiters and endpoint rules, GraphQL directive ordering and@oneOflowering.docs/ir-spec-matrix.mdgains the Erlang/OTP column, six new capability rows (one-way ops, positional encoding, symbol literals, unsolicited messages, multi-format payload schemas, extension ranges), and ~25 cell corrections (e.g. Smithy optionality-vs-nullability was overstated, protobufreservedis not versioning, GraphQL has no server construct).architecture.mdandCLAUDE.mdadd OTP to the format lists.Test plan
MessageRef,ReplyTo,FilenameParam, bool XML node flags) remain.